Responses to comments and elaborations of previous posts III

This post is dedicated to the memory of Rabbi Chaim Flom, late rosh yeshiva of Yeshivat Ohr David in Jerusalem. I first met Rabbi Flom thirty years ago when he became my teacher at the Hebrew Youth Academy of Essex County (now known as the Joseph Kushner Hebrew Academy; unfortunately, another one of my teachers from those years also passed away much too young, Rabbi Yaakov Appel). When he first started teaching he was known as Mr. Flom, because he hadn't yet received semikhah (Actually, he had some sort of semikhah but he told me that he didn't think it was adequate to be called "Rabbi" by the students.) He was only at the school a couple of years and then decided to move to Israel to open his yeshiva. I still remember his first parlor meeting which was held at my house. Rabbi Flom was a very special man. Just to give some idea of this, ten years after leaving the United States he was still in touch with many of the students and even attended our weddings. He would always call me when he came to the U.S. and was genuinely interested to hear about my family and what I was working on. He will be greatly missed

Designing a commutative replicated data type

Commuting operations greatly simplify consistency in distributed systems. This paper focuses on designing for commutativity, a topic neglected previously. We show that the replicas of \emph{any} data type for which concurrent operations commute converges to a correct value, under some simple and standard assumptions. We also show that such a data type supports transactions with very low cost. We identify a number of approaches and techniques to ensure commutativity. We re-use some existing ideas (non-destructive updates coupled with invariant identification), but propose a much more efficient implementation. Furthermore, we propose a new technique, background consensus. We illustrate these ideas with a shared edit buffer data type

Persistent Memory Programming Abstractions in Context of Concurrent Applications

The advent of non-volatile memory (NVM) technologies like PCM, STT, memristors and Fe-RAM is believed to enhance the system performance by getting rid of the traditional memory hierarchy by reducing the gap between memory and storage. This memory technology is considered to have the performance like that of DRAM and persistence like that of disks. Thus, it would also provide significant performance benefits for big data applications by allowing in-memory processing of large data with the lowest latency to persistence. Leveraging the performance benefits of this memory-centric computing technology through traditional memory programming is not trivial and the challenges aggravate for parallel/concurrent applications. To this end, several programming abstractions have been proposed like NVthreads, Mnemosyne and intel's NVML. However, deciding upon a programming abstraction which is easier to program and at the same time ensures the consistency and balances various software and architectural trade-offs is openly debatable and active area of research for NVM community. We study the NVthreads, Mnemosyne and NVML libraries by building a concurrent and persistent set and open addressed hash-table data structure application. In this process, we explore and report various tradeoffs and hidden costs involved in building concurrent applications for persistence in terms of achieving efficiency, consistency and ease of programming with these NVM programming abstractions. Eventually, we evaluate the performance of the set and hash-table data structure applications. We observe that NVML is easiest to program with but is least efficient and Mnemosyne is most performance friendly but involves significant programming efforts to build concurrent and persistent applications.Comment: Accepted in HiPC SRS 201

CRDTs: Consistency without concurrency control

A CRDT is a data type whose operations commute when they are concurrent. Replicas of a CRDT eventually converge without any complex concurrency control. As an existence proof, we exhibit a non-trivial CRDT: a shared edit buffer called Treedoc. We outline the design, implementation and performance of Treedoc. We discuss how the CRDT concept can be generalised, and its limitations